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Executive  Summary 


Statement  of  objectives 

The  research  carried  out  under  this  grant  concerned  the  development  of  improved  modeling  and 
control  techniques.  The  techniques  are  built  upon  observational  data  and  are  tailored  to  the  problem 
at  hand.  The  models  used  are  global  discrete  time  mappings,  ordinary  and  partial  differential 
equations. 

One  major  effort  of  the  proposed  program  was  the  exploitation  of  symmetries  to  constrain  the 
fitted  models.  These  models  were  shown  to  be  more  compact  and  could  accurately  model  behaviors 
not  explicitly  shown  in  the  data. 

A  second  major  effort  was  an  examination  of  synchronization  between  identical  dynamical  sys¬ 
tems.  The  focus  has  been  on  developing  a  method  for  designing  coupling  schemes  that  guarantee 
synchronous  motion  between  the  systems.  The  results  are  analytical  and  represent  a  method  for 
determining  the  complete  state  of  a  nonlinear  system  from  limited  measurements. 

A  third  major  effort  concerned  the  development  of  symbolic  time  series  methods.  This  technique 
has  great  promise  in  high-noise  situations,  where  it  has  been  shown  to  be  capable  of  robust  param¬ 
eter  estimation  for  both  low-  and  high-dimensional  systems.  During  the  period  of  this  project,  we 
concentrated  on  developing  a  theory  of  symbolic  time  series  analysis  for  Markov  systems,  examined 
the  effects  on  the  symbolic  data  of  variation  of  both  the  time  delay  for  sampling  and  the  partition, 
as  well  as  initiating  the  investigation  of  detecting  of  changes  of  state. 

Methods  employed 

The  models  used  are  expansions  in  polynomials,  or  have  been  derived  from  first  principles.  The  basis 
set  used  for  the  expansion  is  constructed  to  be  orthonormal  on  the  measured  data  after  embedding 
into  an  appropriate  state  space.  The  coefficients  of  the  models  are  fit  to  the  data  by  either  a  least 
squares,  or  an  annealing  procedure  using  the  raw  data  or  its  coarse-grained  symbolic  form.  In 
addition,  the  fitting  procedure  accounts  for  the  size  and  complexity  of  the  expansion  models.  This 
results  in  models  that  are  optimal  in  the  sense  that  they  are  the  simplest  models  (within  a  given 
class  of  models)  that  are  consistent  with  the  data. 

An  important  and  novel  aspect  of  the  research  program  is  the  array  of  tests  and  constraints 
implemented  to  ensure  that  the  models  are  corret  and  contain  the  right  physics.  In  order  to  insure 
that  the  models  are  close  approximations  to  the  true  equations  of  motion  in  the  reconstructed  phase 
space  a  series  of  a  priori  constraints  and/or  a  posteriori  tests  are  imposed.  The  a  priori  constraints 
involve  determining  from  the  data  any  symmetries  the  attractor  may  exhibit  and  restricting  the  class 
of  models  to  those  that  respect  the  symmetries.  The  a  posteriori  test  compares  the  coarse-grained 
symbol  statistics  generated  by  the  model  with  that  of  the  data. 

Significance  of  the  proposed  activity 

The  proposed  research  will  significantly  enhance  the  ability  to  detect,  model  and  control  the  dynamics 
of  low-dimensional  nonlinear  systems  using  observed  time  series  data.  The  synchronization  of  two 
nonlinear  oscillators  can  be  exploited  as  a  means  of  nondestructive  testing  of  devices,  or  as  a  real  time 
monitor  of  dynamics,  or  as  a  mechanism  for  controlling  dynamics.  Symbolic  time  series  methods  are 
attractive  because  they  are  relatively  robust  to  noise  and  involve  only  discrete  calculations.  Hence, 
they  should  be  very  fast  in  real-time  applications.  The  approach  used  is  comprehensive  and  will  be 
implemented  on  experimentally  obtained  data  from  a  diverse  group  of  sources. 
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C.  INTRODUCTORY  COMMENTS: 


The  project  covered  by  this  report  (AFOSR  Grant  No.  F49620-96-1-011)  was  carried  out  during 
the  time  frame  April  1,  1996-March  31,  1997.  Part  of  this  period  overlapped  the  time  frame  of  the 
earlier  effort  covered  by  Grant  No.  AFOSR  F49620-95-1-0261  (final  report  dated  Dec.  19,  1996)  due 
to  a  no-cost  extension  granted  on  the  earlier  project. 

In  the  interests  of  completeness  of  exposition  we  have  included  a  brief  summary  of  the  earlier  work 
as  well,  as  the  present  final  report  builds  directly  on  the  earlier  results.  New  results,  developed  under 
Grant  No.  F49620-96-1-011,  are  clearly  indicated  in  the  text  and  summarized  in  Section  G. 

D.  RESEARCH  OBJECTIVES 

The  goal  of  this  research  program  is  the  development  of  new  robust  algorithms  for  diagnosing, 
modeling,  and  controlling  nonlinear  systems.  The  focus  was  on  systems  that  have  only  a  few  degrees 
of  freedom,  but  some  of  the  techniques  developed  have  wider  applicability.  (In  this  context  the 
number  of  degrees  of  freedom  is  equal  to  the  number  of  independent  variables  needed  to  accurately 
model  the  system  which  generated  the  data.) 

Time-series  data,  in  conjunction  with  whatever  a  priori  information  is  known  about  the  system,  are 
used  as  input  to  the  modeling  process.  These  methods  are  robust  to  uncertainties  in  the  models,  and 
to  the  presence  of  noise  [1-5].  These  techniques  have  been  successfully  applied  to  data  from  chemical 
reactions,  electronic  circuits,  and  mechanical  systems  [1].  Other  researchers  are  also  beginning  to 
apply  these  methods  to  experimental  data  [6-8]. 

E.  BACKGROUND 

Successful  modeling  relies  on  mathematical  results  from  Mane  [9],  Takens  [10],  Sauer,  Yorke,  and 
Casdagli  [11],  and  others  who  have  shown  that  it  is  often  possible  to  reconstruct  the  full  multidimen¬ 
sional  dynamics  of  a  nonlinear  system  from  a  single  scalar  time  series,  s{n)  =  s(nr)  for  n  =  1, 2, . . . 
(r  is  the  sampling  interval  associated  with  the  measurements)  [12].  The  most  common  technique 
uses  time  delays  to  form  vectors  y(n) 

y(n)  =  [s{n),  s{n  +  T),  . . .  ,s{n  +  {d  -  1)T)], 

where  y(n)  €  M^.  Time  evolution,  as  given  by  y(n)  h->  y(n  -i- 1),  is  equivalent  (diffeomorphic) 
to  the  true  evolution  of  the  system  that  produced  the  scalar  time  series.  Fraser  and  Swinney  [13] 
have  developed  a  technique  for  determining  T  from  data,  while  Kennel  et  al.  [14]  have  developed  a 
technique  for  determining  d. 


1.  Low- dimensional  modeling 

The  dynamics  is  modeled  by  global  discrete-time  maps 

y(n  +  1)  =  F[y(n)], 

or  global  ordinary  differential  equations  (ODE’s) 

I  = 
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When  modeling  the  observed  system  via  ODE’s,  time  evolution  of  the  data  is  modeled  as  a  single 
implicit  Adams  integration  time  step 

M 

y(n  +  1)  =  y(n)  +  r  ai^^F[y (n  +  1  -  j)]. 

j=0 

The  are  the  Adams  integration  coefficients,  and  are  known  for  all  values  of  j  and  M. 

For  a  generic  time  series  one  does  not  know  the  functional  form  of  F.  Hence,  the  best  that  one 
can  hope  for  is  a  series  expansion  in  some  set  of  basis  functions, 

Np 

1=0 

The  data  approximates  an  invariant  distribution  on  the  attractor  that  defines  the  system  under 
observation 


71=1 

where  N  is  the  number  of  data  vectors.  The  basis  functions  are  polynomials  constructed 

(via  Gram-Schmidt)  to  be  orthonormal  on  the  attractor  defined  by  the  data, 

—  j  dz  p{z) 

=  5ij. 

In  this  equation  I  and  J  are  index  vectors  used  to  indicate  the  order  of  the  polynomials. 

It  has  been  shown  that  this  formulation  of  the  modeling  problem  allows  more  accurate  and  robust 
determination  of  F,  for  larger  sampling  intervals,  than  previous  methods  [1,7,8,15-17]. 


2.  Symbolic  time  series  analysis 


In  symbolic  time-series  analysis  the  state  space  of  the  system  is  partitioned  into  a  finite  number 
of  cells  and  a  symbol,  s,  is  assigned  to  each  cell.  Such  a  symbolic  approach  is  appealing  because 
the  symbol  statistics  are  robust  in  the  presence  of  noise  and  easy  to  estimate  from  observations. 
Our  previous  work  [4]  showed  that  the  symbolic  data  are  quite  robust  and  that  even  at  high  noise 
levels  (signal/noisew  0(1))  the  effects  of  noise  on  the  symbolic  data  can  be  tracked  and  effectively 
eliminated. 

To  briefly  summarize  the  approach,  we  convert  the  observed  analog  signal  stream,  Xn,  into  a  symbol 
sequence  by  passing  it  through  a  threshold  function  which  takes  {xn}  =  {xi,X2,  -••,xn)  ^  {s„}  = 
(si,  S2,  •  •  • ,  Sn)  where  Sk  G  (a,  b).  For  example,  if  a;„  <  x*  then  s„  =  a  and  if  Xri  >  x*  then  s„  =  b. 
From  the  symbol  sequence,  {s„},  construct  the  symbol  tree 


Pa  Pb 

Paa  Pab  Pba  Pbb 

Paaa  Paab  Paba  Pabb  Pbaa  Pbab  Pbba  Pbba 

etc. 

Here  Paab  is  the  probability  of  observing  the  sequence  aab,  etc.  The  symbol  tree  is  a  compact 
summary  of  (coarse  grained)  information  about  multiple-time-step  correlations  in  the  signal.  The 
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Noisy  Lorenz  Attractor  Noisy  Kmbedded  Lorenz  Attractor 


FIG.  1. 


higher  levels  of  the  tree  are  determined  by  the  lowest  level  measured,  hence  we  focus  only  on  the 
bottom  level  used  (typically  going  to  level  5  has  proven  to  be  sufficient  for  most  purposes).  In  what 
follows,  we  denote  the  vector  formed  by  the  probabilities  at  the  lowest  level  by  P,  e.g.  at  level  5: 
P  =  (Paaaaa,  ■  ■  ■  ,Pbbbbb)  ^  Not  all  of  these  quantities  are  independent,  as  will  be  discussed  in 
Section  F  5. 


F.  RESULTS 

1.  Using  symmetries  to  constrain  expansion  models 

Symmetries  represent  an  important  class  of  physical  properties  that  can  be  used  to  constrain  models 
of  dynamical  systems.  Using  a  symmetry  to  constrain  a  model  means  restricting  the  functional  form 
of  the  model  to  one  that,  a  priori,  contains  the  symmetry.  They  can  be  determined  from  the  data 
and/ or  from  first  principles.  We  have  found  that  if  a  symmetry  is  present  in  the  data  or  the  dynamics 
then  an  equivariante  model  constructed  from  the  data  accurately  models  the  dynamics  [2] . 

To  illustrate  how  symmetries  constrain  a  model,  suppose  the  attractor.  A,  is  invariant  under  the 
action  of  the  symmetry  S  (So  A  =  A).  This  condition  results  in  the  following  equivariance  condition 
on  F 


F(Soy)  =  SoF(y).  (1) 

Clearly,  Eq.  (1)  places  restrictions  on  the  functional  form  of  F. 

The  Lorenz  system  is  a  dynamical  system  where  we  used  symmetry  to  restrict  the  functional  form 
of  F.  Figure  1(a)  shows  a  noisy  representations  of  the  true  Lorenz  attractor.  Despite  the  noise. 
Fig.  1(a)  indicates  that  if  S  is  a  rotation  about  the  z  axis  by  tt  radians  then  S  o  ^4  =  .4.  The  origin 
of  this  symmetry  is  the  x  —x  and  y  -y  invariance  of  the  Lorenz  equations.  For  this  system 
Eq.  (1)  implies  that  all  terms  proportional  to  x”'y'^  must  vanish  for  even  values  of  n  +  m.  Thus, 
on  the  basis  of  equivariance  half  of  the  terms  in  the  model  can  be  eliminated.  Of  course,  if  the 
data  y  =  [x,  y,  z]  is  noise  free  then  we  expect  the  coefficients  of  these  terms  to  vanish.  However, 
experimental  data  is  never  noise  free.  Therefore,  it  is  best  to,  a  priori,  eliminate  terms  from  the 
model. 

Figure  1(b)  shows  a  Lorenz  attractor  that  has  been  reconstructed  by  embedding  a  scalar  time 
series  given  by  the  x  coordinate  of  the  data  shown  in  Fig.  1(a).  The  time  series  is  symmetric  about 
zero  which  leads  to  the  inversion  symmetry  (S  =  -1)  evident  in  Fig.  1(b).  Using  an  ensemble 
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FIG.  2.  Graphs  of  parameter  uncertainty  versus  the  index  associated  with  the  parameter. 


of  data  sets  we  constructed  models  without  the  restriction  of  Eq.  (1).  The  results  are  shown  in 
Fig.  2,  where  is  the  mean  value  of  the  parameter  and  is  the  standard  deviation  of  the 

parameter  about  its  mean  value.  The  circles  squares  and  diamonds  indicate  the  first,  second,  and 

third  components  of  respectively.  The  solid  line  represents  I  =  2- 

For  symbols  above  this  line  |  (p(*))  |  ±or(b  straddles  zero.  Hence,  the  value  of  p(*)  obtained  by  the 
fitting  procedure  is  statistically  indistinguishable  from  zero  and  one  conjectures  that  their  values  are 
dominated  by  noise,  finite  sample  size,  and  round  off  effects.  This  is  supported  by  noticing  that  all 
of  the  symbols  for  index  values  0,  4-9,  and  20-34  are  above  the  line.  These  indices  correspond  to 
terms  in  F  whose  coefficients,  should  vanish  because  their  basis  function,  is  not  equivariant 
under  S  =  —1. 

For  comparison  we  also  estimated  the  uncertainty  of  the  p(*)’s  using  standard  statistical  meth¬ 
ods  [2].  These  uncertainties,  were  found  to  be  good  estimates  of  Thus,  in  some  cases,  it  is 
possible  to  detect  the  presence  of  a  symmetry  from  a  single  data  set  by  examining  I  [2]- 

We  also  examined  data  from  the  electronic  circuit  shown  in  Fig.  3(a).  It  is  possible  to  determine 

from  first  principles  that,  for  certain  values  of  a,  the  dynamics  lives  on  one  of  the  two  disjoint 

attractors  shown  in  Fig.  3(b)  [18].  For  this  value  of  a  a  clear  plane  of  symmetry  can  be  seen  between 
the  attractors  and  a  single  experimentally  measured  time  series  would  correspond  to  one  or  the  other 
of  these  attractors,  but  not  both.  As  a  increases  the  two  attractors  merge  into  a  single  attractor  via 
a  symmetry  increasing  bifurcation  (also  called  a  crisis  [19]). 

Our  work  examined  data  from  before  and  after  the  bifurcation.  When  examining  data  taken 
before  the  bifurcation  we  found  that  equivariant  models  constructed  using  data  from  one  of  the 
attractors  accurately  mimics  the  dynamics  on  both  attractors  [2].  This  is  not  the  case  for  previous 
modeling  techniques,  which  require  data  from  both  attractors  and  two  models  [2].  Furthermore, 
the  equivariant  model  contains  fewer  coefficients  than  either  of  the  nonequivariant  models.  When 
modeling  data  taken  after  the  bifurcation  we  found  that  we  were  able  to  detect  the  presence  of  the 
symmetry  using  the  methods  discussed  above  for  the  Lorenz  example. 
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G.  ACCOMPLISHMENTS  AND  NEW  FINDINGS 


The  primary  accomplishments  of  the  grant  period  (4/1/96  -  3/31/97)  are: 

1.  Low-dimensional  modeling 

•  Exploitation  of  symmetries  to  constrain  the  modeling  process.  Modeling  algorithms  were 
developed  which  detect  and  impose  symmetries  on  the  functional  form  of  the  model.  This 
has  been  shown  to  improve  the  stability  of  the  modeling  process,  as  well  as  ensuring  that 
important  properties  of  the  dynamical  system  are  represented  by  the  model  even  if  they 
are  not  directly  present  in  the  data  set  [2]. 

•  A  rigorous  sufficient  criteria  has  been  derived  which  guarantees  linearly  stable  synchroniza¬ 
tion  between  dynamical  systems  when  they  are  coupled  in  a  drive-response  manner.  This 
result  is  an  analytic  method  for  solving  the  observer  problem  for  nonlinear  plants  [3]. 

2.  Symbolic  time  series  analysis 

•  Development  of  a  basic  theory  for  the  symbolic  time  series  analysis  of  Markov  systems. 

•  Demonstration  that  the  symbolic  approach  can  be  used  to  develop  robust  esimates  of 
correlation  times  and  detect  weak  periodic  signals  masked  by  noise  or  chaotic  signals. 
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2.  R.  Brown,  V.  In  and  E.  R.  Tracy,  “Parameter  uncertainties  in  models  of  equivariant  dynamical 
systems”,  Physica  D  102,  ???  (1997). 

3.  X.-Z.  Tang,  E.  R.  Tracy  and  R.  Brown,  “Symbol  statistics  and  spatio-temporal  systems”,  Phys¬ 
ica  D  102,  253  (1997). 

4.  R.  Brown,  “Using  Models  to  diagnose,  test,  and  control  chaotic  systems”,  to  appear  in  the 
proceedings  of  the  Third  Experimental  Chaos  Conference. 

5.  G.  Gouesbet,  L.  Le  Sceller,  C.  Letellier  and  R.  Brown,  “Reconstruction  of  a  set  of  equations 
from  scalar  time  series” ,  to  appear  in  the  proceedings  of  the  Eleventh  Annual  Florida  Workshop 
on  Nonlinear  Astronomy. 

6.  E.  R.  Tracy,  “Toward  a  theory  of  symbolic  time  series  analysis”,  in  preparation. 

7.  X.-Z.  Tang  and  E.  R.  Tracy,  “Data  compression  and  information  retrieval  via  symbolization”, 
in  preparation. 


J.  INTERACTIONS/TRANSITIONS 


1.  Presentations  at  meetings 

1.  Poster  presentation  (contributed)  at  Sherwood  (Fusion  Theory)  Meeting,  Philadelphia,  PA 
(March,  1996),  E.  Tracy,  X.-Z.  Tang,  R.  Brown  k  S.  Burton. 

2.  Poster  presentation  (contributed)  at  American  Physical  Society,  Division  of  Plasma  Physics 
Meeting,  Denver,  CO  (November,  1996),  E.  Tracy. 

3.  Seminars  at  AlliedSignal  Engines,  Phoenix,  AZ,  March,  1996,  R.  Brown  and  E.  Tracy. 
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2.  Transitions 


1.  Wm  &  Mary/AlliedSignal: 

Performer: 

Professors  E.  R.  Tracy  and  Reggie  Brown,  Dr.  X.-Z.  Tang  &  Ms.  Sharon  Burton. 

Telephone  (Tracy):  (757)221-3527. 

Customer: 

AlliedSignal  Inc. 

Microelectronics  &  Technology  Center 
9140  Old  Annapolis  Road/MD  108 
Columbia,  MD  21045 
Contact: 

Dr.  R.  Burne 
Research  Manager 
(410)964-4159. 

Anticipated  result:  Using  the  symbol  statistics  to  detect  transitions  in  complex  systems,  e.g. 
noise-driven  turbulent  flows. 

Application:  Early  detection  of  rotating  stall  in  turbines. 

Ms.  Burton  was  supported  by  AlliedSignal  MTC  as  an  intern  during  the  summer  of  1996. 

2.  Oak  Ridge  National  Lab/Ford  Motor  Company: 

Work  performed  as  part  of  a  Cooperative  Research  and  Development  Agreement  (CRADA),  number 
ORNL-95-0337  titled  “Engine  Control  Improvement  Through  Application  of  Chaotic  Time  Series 
Analysis” . 

Performer: 

Dr.  C.  Stuart  Daw 

Oak  Ridge  National  Laboratory 

P.O.  Box  2009 

Oak  Ridge,  TN  37831-8088 

Telephone:  (615)574-0373. 

Customer: 

Ford  Motor  Co. 

Dearborn,  MI 
Contact: 

Dr.  John  Hoard 
(313)594-1316. 

Anticipated  result:  Using  symbol  statistics  to  do  parameter  fitting  for  an  internal  combustion 
engine  model  [6].  (Aspects  of  the  work  are  subject  to  a  patent  disclosure.) 

Application:  Improved  feedback  control  for  internal  combustion  engines  to  reduce  NOx  and  hy¬ 
drocarbon  emission  and  increase  fuel  efficiency. 
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